Image processing method

ABSTRACT

Low-pass filtering is applied only to the borders of an original mage to reduce the noise in the borders of image, thereby preventing errors in extrapolation. Then, an extended region is provided around the image by extrapolation, thereby eliminating the need for border processing in the borders of the image when performing the final low-pass filtering. Finally, low-pass filtering is applied to the regions where the original image exists. Thus, a filtering effect can be obtained in which the regions nearer to the center of the image and the border regions of the image have the same characteristics.

FIELD OF THE INVENTION

The present invention relates to an image processing method whichapplies low-pass filtering to a captured image.

BACKGROUND OF THE INVENTION

Image processing methods which apply low-pass filtering to imagestypically obtain the average of pixels in a neighborhood of a pixelunder consideration and use the average as a new brightness value of thepixel under consideration. Here, widening the area over which averagingis performed can enhance low-pass filtering and therefore remove localfeatures in the image and provide a macro brightness distribution.

Conventional image processing methods are used primarily for the purposeof removing noise. Accordingly, they are designed to preserve edgecomponents of pixels to be processed as much as possible. For example,one method used is to select only pixels located in the direction inwhich a stronger correlation exists when neighborhood pixel averaging isperformed (see for example Japanese Patent Laid-Open No. 2001-61157).

However, none of the conventional image processing techniques providesarrangements for processing in the borders of images. For example, iflow-pass filtering is performed by averaging 3 pixels located in allvertical and horizontal directions from a pixel of interest, averagingof a 7×7 pixels centered at that pixel is required. If a pixel ofinterest is in the right border of an image, however, there are nopixels to the right of that pixel. Therefore, in some typical methods,the original brightness values in the 3-pixel-wide borders of an imageare used without change; in other typical methods, new brightness valuesare computed from the average values only in the areas in which pixelsexist.

Using the original brightness values without change has a problem thatthe borders are not low-pass-filtered, of course. On the other hand,computing a new brightness value from the average only of the areaswhere pixels exist has problems that the strength of a low-pass filtervaries from location to location and that wrong brightness levels areassigned. The reason why wrong brightness levels are assigned is thatif, for example, the brightness of an image increases from left toright, a new brightness value for a pixel of interest that is located inthe right border is computed from the average of the brightness valuesof the pixel of interest and of pixels located on the left of the pixelof interest (the average of only the pixels having brightness valuesless than that of the pixel of interest) and consequently the pixel willhave a brightness value lower than the proper brightness value at thatposition.

The present invention solves these problems with the conventionaltechniques and an object of the present invention is to provide an imageprocessing method and an image processing apparatus capable of applyingoptimum low-pass filtering that ensures an average brightness even forpixels in borders.

DISCLOSURE OF THE INVENTION

According to the first aspect of the present invention, there isprovided an image processing method of treating an image astwo-dimensional data of pixels having multiple gray levels, the methodincluding a step of applying low-pass filtering to at least the pixelsin the borders of an original image or pixels near the borders, a stepof extending the region of the original image and obtaining thebrightness of the extended portion by extrapolation using as an origin apixel included in the region to which the low-pass filtering has beenapplied, and a step of applying low-pass filtering to the region of theoriginal image by using the brightness of the extended portion.

According to the second aspect of the present invention, in the imageprocessing method of the first aspect, the step of applying low-passfiltering to at least the pixels in the borders of an original image orpixels near the borders includes applying one-dimensional low-passfiltering to the pixels in the borders of the original image.

According to the third aspect of the present invention, in the imageprocessing method of the first aspect, the step of applying low-passfiltering to at least the pixels in the borders of an original image orpixels near the borders includes applying low-pass filtering by usingpixels located in the direction perpendicular to the direction ofextrapolation.

According to the fourth aspect of the present invention, in the imageprocessing method of the first aspect, the extent to which the region ofthe original image is extended is determined depending on the extent oflow-pass filtering to be applied to the region of the original image byusing the brightness of the extended portion.

According to the fifth aspect of the present invention, in the imageprocessing method of the fourth aspect, if the size of a neighborhoodpixel region required for computation in low-pass filtering applied tothe region of the original image including the extended portion is Apixels high×B pixels wide, the region of the original image is extendedby A pixels in the vertical direction and by B pixels in the horizontaldirection.

According to the sixth aspect of the present invention, the imageprocessing method of the first aspect further includes a step ofobtaining the differences between a first processed image obtained byperforming the image processing method according to claim 1 and anoriginal image to obtain a second processed image.

According to the seventh aspect of the present invention, in the imageprocessing method of the sixth aspect, the second processed image isbinarized to detect a defective portion contained in the original image.

According to the eighth aspect of the present invention, there isprovided an image processing apparatus including an image processingunit which averages an original image which is an image of a flat panelused in inspection and captured by a camera to be inspected and detectsa first processed image which is very close to the brightnessdistribution of the original image, a subtracter which detects a secondprocessed image of the differences between the first processed image andthe original image, a binarizing circuit which detects a resultant imageresulting from binarizing of the second processed image, and adetermining circuit which compares the resultant image against anallowable-quality standard to determine whether the quality of thecamera to be inspected is acceptable or not.

According to the ninth aspect of the present invention, there isprovided an image processing method including a step of applyinglow-pass filtering to at least the pixels in the borders or pixels nearthe borders of an original image which is an image of a flat panel usedin inspection and captured by a camera to be inspected and detecting afirst processed image which is very close to the brightness distributionof the original image, a step of detecting a second processed image of adifference between each pixel in the first processed image and itscorresponding pixel in the original image, a step of applying medianfiltering to the second processed image and detecting a binarizedresultant image, and a step of comparing the total count of defectpixels contained in the resultant image with the number of defect pixelsspecified in a quality standard and determining whether the camera isdefective or not.

According to the tenth aspect of the present invention, there isprovided an image processing method of treating an image astwo-dimensional data of pixels having multiple gray levels, the methodincluding a step of applying low-pass filtering to at least the pixelsin the borders of an original image or pixels near the borders, a stepof obtaining a first processed image by extending the region of theoriginal image and obtaining the brightness of the extended portion byextrapolation using as an origin a pixel included in the region to whichthe low-pass filtering has been applied, a step of obtaining a secondprocessed image by applying low-pass filtering to the first processedimage, and a step of generating an image by comparing each pixel in thesecond processed image with its corresponding pixel in the originalimage and performing low-pass filtering using only the pixels of thefirst processed image at locations where differences obtained by thecomparison between the pixels are smaller than a predeterminedthreshold.

According to the eleventh aspect of the present invention, the imageprocessing method of the tenth aspect further includes a step ofcorrecting an image to be corrected which is different from the originalimage by using a calibration image which is the image outputted from thestep of generating an image by performing low-pass filtering using onlythe pixels of the first processed image at locations in the originalimage where differences obtained by the comparison between the pixelsare smaller than a predetermined threshold.

According to the twelfth aspect of the present invention, in the imageprocessing method of the eleventh aspect, division is performed betweenan image other then the original image and the calibration image on apixel-by-pixel basis to correct an image different from the originalimage.

According to the thirteenth aspect of the present invention, there isprovided an image processing method including a step of applyinglow-pass filtering to at least the pixels in the borders of an originalimage or pixels near the borders, a step of obtaining a first processedimage by extending the region of the original image and obtaining thebrightness of the extended portion by extrapolation using as an origin apixel included in the region to which the low-pass filtering has beenapplied, a step of applying low-pass filtering to the first processedimage to obtain a second processed image, and a step of comparing eachpixel in the second processed image with its corresponding pixel in theoriginal image and applying further low-pass filtering to the firstprocessed image without using pixels having differences greater than orequal to a predetermined threshold.

According to the fourteenth aspect of the present invention, in theimage processing method of the thirteenth aspect, the step of comparingeach pixel in the second processed image with its corresponding pixel inthe original image and applying further low-pass filtering to the firstprocessed image without using pixels having differences greater than orequal to a predetermined threshold, includes a step of comparing eachpixel in the second processed image with its corresponding pixel in theoriginal image and if the difference in brightness is greater than apredetermined value, marking the relevant pixel, and a step of settingthe marked pixel as a pixel of interest, checking the values of pixelsin a neighborhood of the pixel of interest that are used in low-passfiltering, and using the values for averaging to set the average as anew brightness value of the pixel of interest.

According to the image processing method of the present invention, anoptimum low-pass filtering result can be provided in which the averagebrightness of pixels even near the borders of an image is guaranteed.

Furthermore, the image processing apparatus using the image processingmethod of the present invention can implement an apparatus forinspecting cameras that is capable of detecting defects even in theborders of an image without fail and also can implement a fluorescentmicroscope capable of providing an image free of inconsistencies inbrightness.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a flowchart of a process in an image processing methodaccording to a first embodiment of the present invention;

FIG. 2 is a diagram showing an arrangement of pixels in an originalimage in the first embodiment;

FIG. 3 is a diagram showing an arrangement of pixels in an intermediateimage in the first embodiment;

FIG. 4A is a graph in which the heights represent brightness values ofthe pixels in the third row from the top of the original image shown inFIG. 2;

FIG. 4B is a graph in which the heights represent brightness values ofthe pixels in the fifth row from the top of the image shown in FIG. 3;

FIG. 5 is a block diagram of an image processing apparatus according toa second embodiment of the present invention;

FIG. 6 is a block diagram of a fluorescent microscope imaging systemaccording to a third embodiment of the present invention;

FIG. 7 is a flowchart of a process for generating a calibration imageaccording to the third embodiment;

FIG. 8 is a flowchart of step 704 in the third embodiment; and

FIG. 9 is a flow chart of software implementation of a defect detectingapparatus 502 according to the second embodiment of the presentinvention.

DESCRIPTION OF THE EMBODIMENTS

An image processing method according to the present invention will bedescribed below with respect to specific embodiments thereof.

First Embodiment

FIGS. 1 to 4A and 4B show a first embodiment of the present invention.

FIG. 1 shows a process flow of an image processing method according tothe first embodiment.

At step 101, low-pass filtering is applied only to the pixels located inthe borders of an original image P0 of interest.

At step 102, extrapolation is performed to generate an image of a sizelarger than the original image.

At step 103, low-pass filtering is applied to the entire original imageto generate a desired low-pass-filtered image.

The above-stated procedure will be descried with respect to a morespecific example.

FIG. 2 shows conceptual diagrams showing the pixels in an original imageand the pixels resulting from the processing at step 101. In FIG. 2,reference numeral 201 indicates the original monochrome image in whichthe pixels have brightness values in a 256-level gray scale, rangingfrom 0 to 255. For simplicity, it is assumed that the image is 9 pixelswide×7 pixels high. Also, the brightness value of each pixel isrepresented by A(x, y).

A procedure of low-pass filtering at step 101 will be described first,in which the low-pass filtering is applied only to the pixels located inthe borders of the original image 201.

Here, low-pass filtering will be applied only to the pixels representedas shaded portions. First, low-pass filtering is horizontally applied toA(1, 0)-A(7, 0). The horizontal low-pass filtering basically computesthe average of the brightness values of 5 pixels in the original image,centered on a pixel A(x, y) of interest and sets the average value as anew brightness value B(x, y) of the pixel of interest. For example, ifA(4, 0) is the pixel of interest, the value(A(2, 0)+A(3, 0)+A(4, 0)+A(5, 0)+A(6, 0))/5is computed and the result is set as the new brightness value B(4, 0).When A(1, 0) is the pixel of interest, the second pixel from the pixelA(1, 0) on the left does not exist, therefore the nonexistent pixel isnot included in the low-pass filter, therefore the value(A(0, 0)+A(1, 0)+A(2, 0)+A(3, 0))/4is set as the new brightness value B (1, 0). Similarly, when A(7, 0) isthe pixel of interest, then the value(A(5, 0)+A(6, 0)+A(7, 0)+A(8, 0))/4is set as the new brightness value B(7, 0). Horizontal low-passfiltering is applied to A(1, 6) to A(7, 6) in exactly the same way.

For A(1, 0) to A(0, 5), low-pass filtering is performed vertically. Thevertical low-pass filtering computes the average of the originalbrightness values of 5 pixels in the original image, centered on a pixelA(x, y) of interest and sets the average value as a new brightness valueB(x, y) of the pixel of interest. For example, if A(0, 2) is the pixelof interest, then the value(A(0, 0)+A(0, 1)+A(0, 2)+A(0, 3)+A(0, 4))/5is computed and is set as the new brightness value B(0, 2). When A(0, 1)is the pixel of interest, the second pixel above the pixel of interestA(0, 1) does not exit, therefore the nonexistent pixel is not includedin the low-pass filter and the value(A(0, 0)+A(0, 1)+A(0, 2)+A(0, 3))/4is set as the new brightness value B(0, 1). Vertical low-pass filteringis also applied to A(8, 1-A(8, 5) in exactly the same way.

When A(0, 0), A(8, 0), A(0, 6), and A(8, 6) are the pixel of interest,low-pass filtering is applied both in the vertical and horizontaldirections because they are located in a corner. Accordingly, when A(0,0) is the pixel of interest, the value(A(0, 0)+A(1, 0)+A(2, 0)+A(0, 1)+A(0, 2)) /5is computed and set as the new brightness value B(0, 0). When A(8,0) isthe pixel of interest, then the value(A(6, 0)+A(7, 0)+A(8, 0)+A(8, 1)+A(8, 2))/5is computed and set as the new brightness value B(8, 0). Similaroperations are performed on A(0, 6) and A(8, 6).

On the completion of the process described above, an image 202 resultingfrom the low-pass filtering that has been applied only to the pixelslocated in the borders is resulted.

In the foregoing description, the low-pass filter used calculates theaverage of the brightness values of 5 pixels centered on a pixel ofinterest. However, a low-pass filter that computes the average of alarger number of pixels may be used or a low-pass filter that computes aweighted average that is dependent on locations may be used to obtain asimilar effect.

The detail of step 102 will be described below with reference to FIG. 3.

FIG. 3 is a conceptual diagram showing the pixels in an image on whichthe processing at step 102 is applied. The image 301 is generated by theoperation at step 102, in which the white portion is generated by theprocessing at step 101 (which is the image 202 in which only the pixelsin the borders were lowpass-filtered) and the shaded portion representsthe pixels newly generated by the processing at step 102.

That is, the size of the white portion is the same as that of theoriginal image and the shaded portion is an extended portion. In thewhite portion, the locations represented by A(x, y) have the samebrightness value as that in the original image and the locationsrepresented by B(x, y) have the new brightness values resulting from theprocessing at step 101. The width of the extended portion depends on thearea required by a low-pass filter used at step 103.

Here, the low-pass filter used at step 103 computes the average over a5×5 pixel area centered on a pixel of interest and sets the averagevalue as the new brightness value of the pixel of interest. In order toobtain the result of processing at the location B(0, 0), for example,with high accuracy, the brightness values of two pixels located abovethe location and two pixels on the left of the location are required.Therefore, 2-pixel-wide extended region is provided around the originalimage in the first embodiment. The brightness values in the extendedregion are obtained by extrapolation.

How to compute the brightness values in the upper and lower extendedregions will be detailed first.

For example, the brightness value of C(0, −1) is computed by “2*B(0,0)−B(0, 1)” and the brightness value of C(8, −1) is computed by “2*B(8,0)−B(8, 1)”. The brightness value of C(1, −1) is computed by “2*B(1,0)−A(1, 1)”. The brightness values of C(2, −1) to C(7, −1) can becomputed in a similar way. The brightness value of C(0, −2) is computedby “2*B(0, 0)−B(0, 2)” and that of C(8, −2) is computed by “2*B(8,0)−B(8, 2)”. Also, the brightness value of C(1, −2) is computed by“2*B(1, 0)−A(1, 2)”. The brightness values of C(2, −2) to C(7, −2) canbe computed in a similar way.

The brightness value of C(0, 7) is computed by “2*B(0, 6)−B(0, 5) andthat of C(8, 7) is computed by “2*B(8, 6)−B(8, 5). The brightness valueof C(1, 7) is computed by “2*B(1, 6)−A(1, 5)”. The brightness value ofC(1, 7) to C(7, 7) can be computed in a similar way. The brightnessvalue of C(0, 8) is computed by “2*B(0, 6)−B(0, 4)” and that of C(8, 8)is computed by “2*B(8, 6)−B(8, 4)”. The brightness value of C(1, 8) iscomputed by “2*B(1, 6)−A(1, 4)”. The brightness values of C(2, 8) toC(7, 8) can be computed in a similar way.

How to compute the brightness values in the left and right extendedregions will be detailed next.

For example, the brightness value of C(−1, 0) is computed by “2*B(0,0)−B(1, 0)” and that of C(−1, 6) is computed by “2*B(0, 6)−B(1, 6)”. Thebrightness value of C(−1, 1) is computed by “2*(0, 1)−A(1, 1)”. Thebrightness values of C(−1, 2) to C(−1, 5) can be computed in a similarway.

Because the brightness values in the upper and lower extended regions ofthe original image have already been computed, the brightness values ofC(−1, −2) can be computed by using the values of C(0, −2) and C(1, −2)in a similar way. Also, the brightness values of C(−1, −1), C(−1, 7) andC(−1, 8) can be computed in a similar way. Then, the brightness value ofC(−2, 0) is computed by “2*B(0, 0)−B(2, 0)” and that of C(−2, 6) iscomputed by “2*B(0, 6)−B(2, 6). The brightness value of C(−2, 1) iscomputed by “2*B(0, 1)−A(1, 2)”. The brightness values of C(−2, 2) toC(−2, 5) can be computed in a similar way.

Because the brightness values in the upper and lower extended regions ofthe original image have already been computed, the brightness value ofC(−2, −2) can be computed by using the values of C(0, −2) and C(2, −2)in a similar way. Also, the brightness values of C(−2, −1), C(−2, 7) andC(−2, 8) can be computed in a similar way.

Furthermore, the brightness value of C(9, 0) is computed by “2*B(8,0)−B(7, 0)” and that of C(9, 6) is computed by “2*B(8, 6)−B(7, 6)”. Thebrightness value of C(9, 1) is computed by “2*B(8, 1)−A(7, 1)”. Thebrightness values of C(9, 2) to C(9, 5) can be computed in a similarway. Because the brightness values in the upper and lower extendedregions of the original image have already been computed, the brightnessvalue of C(9, −2) can be computed by using the values of C(8, −2) andC(7, −2) in a similar manner. Also, the brightness values of C(9, −1),C(9, 7) and C(9, 8) can be computed in a similar way.

Then, the brightness value of C(10, 0) is computed by “2*B(8, 0)−B(6,0)” and that of C(10, 6) is computed by “2*B(8, 6)−B(6, 6)”. Thebrightness value of C(10, 1) is computed by “2*B(8, 1)−A(6, 1)”. Thebrightness values of C(10, 2) to C(10, 5) can be computed in a similarway. Because the brightness values in the upper and lower extendedregions of the original image have already been computed, the brightnessvalue of C(10, −2) can be computed by using the values of C(8, −2) andC(6, −2) in a similar way. Also, the brightness values of C(10, −1),C(10, 7), and C(10, 8) can be computed in a similar way. Thus, thebrightness values of all pixels in the image 301 have been computed.

Finally, processing at step 103 will be described.

Because a processed image of only the region of the original image isrequired ultimately, computations are performed only on the white regionin FIG. 3. As stated earlier, low-pass filtering performed at step 103involves averaging a 5×5 pixel area centered on a pixel of interest andthe average value is set as a new brightness value of the pixel ofinterest. The brightness value at the pixel location B(0, 0) after theprocessing is the average of the 5×5 pixel area centered on thatlocation, therefore the brightness value can be obtained as

(C(−2, −2) + C(−1, −2) + C(0, −2) + C(1, −2) + C(2, −2) + C(−2, −1) + C(−1, −1) + C(0, −1) + C(1, −1) + C(2, −1) + C(−2, 0) + C(−1, 0) + B(0, 0) + B(1, 0) + B(2, 0) + C(−2, 1) + C(−1, 1) + B(0, 1) + B(1, 1) + B(2, 1) + C(−2, 2) + C(−1, 2) + B(0, 2) + B(1, 2) + B(2, 2))/25.A similar operation is performed on the entire white portion in FIG. 3to obtain the desired processed image. While the exemplary low-passfilter described above computes the average over a 5×5 pixel area, alow-pass filter of a different size or a low-pass filter using adifferent operation such as weighted averaging can be used to obtain asimilar effect.

Because pixels have been generated around the original image by theprocessing at step 102, exceptional processing designed to processingthe borders is not required in the processing at step 103. Instead, thesame processing can be applied to both central and border portions. Theeffect will be described with reference to FIGS. 4A and 4B.

The assumption in the following description is that the brightness in anoriginal image increases from left to right. For simplicity, it isassumed that the brightness of the image in the vertical direction isuniform, and only one-dimensional processing in the horizontal directionwill be described.

FIG. 4A shows the brightness values, represented by height, of thepixels in the third row from the top of the original image shown in FIG.2. The heights of bars 401 to 409 represent the brightness values ofA(0, 2) to A(8, 2), respectively. As can be seen from FIG. 4A, thebrightness increases from left to right. Conventional low-pass filters,which provide the average of 5 pixels centered on a pixel of interest asthe result of their processing, compute the average of the brightnessvalues 407-409 of 3 pixels for the location A(8, 2) because no pixelexists to the right of that location. As can be seen from FIG. 4A, theresult is substantial equal to the height of the brightness value 408,which is lower than the actual, original brightness value 409.

FIG. 4B shows the brightness values in an example in which imageprocessing according to the present invention is applied to the originalimage.

FIG. 4B shows the brightness values of the pixels, represented byheight, in the fifth row from the top of the image shown in FIG. 3. Theheights of bars 401 to 408 represent the brightness values of A(0,2)-A(7, 2), respectively, the height of a bar 410 represents thebrightness value of B(8, 2) and the heights of bars 411 and 412represent the brightness values of C(9, 2) and C(10, 2), respectively.

Brightness value 410, generated by a vertical low-pass filter at step101, is equal to the brightness value 409 because the assumption is madethat there are the equal brightness values in the vertical direction inthe original image. The percentage of increase between the brightnessvalues 411 and 412, generated by extrapolation at step 102, is the sameas that of increase in the brightness values 407 through 410. Processingat step 103 at the location B(8, 2) is averaging of the brightnessvalues 407-412 of 5 pixels. As can be seen from FIG. 4B, the result issubstantially equal to the height of brightness value 410. This showsthat the values close to the average value of the original image can beobtained even in the borders of the image according to the presentinvention.

The brightness values of pixels in the extended regions are obtained byextrapolation at step 102. In the extrapolation, if B(2, 0) for examplehas an error due to noise, the value of C(2, −1) contains an error twiceas large as that error, as can be seen from the equationC(2, −1)=2*B(2, 0)−A(2, 1)Similarly, if C(2, −2) has also an error and the brightness at thelocation B(2, 0) is computed in the processing at step 103, the errorswill have a significant effect on the result of the processing becauseboth C(2, −1) and C(2, −2) are used. This shows that, according to thepresent invention, errors in the brightness values in the borders of anoriginal image due to noise significantly affect the final processingoutcome. This is avoided by the processing at step 101, in whichlow-pass filtering is applied to the pixels in the borders of anoriginal image to prevent noise.

Thus, a low-pass filter capable of preserving the average brightness ofan original image can be implemented.

As has been described, according to the first embodiment, extendedregions are provided around an image before low-pass filtering, therebyenabling the low-pass filtering to be performed in the border portionsof the image under the same conditions as in the central portions of theimage, producing an optimum low-pass filtering outcome in which theaverage brightness even for the pixels in the borders are guaranteed.

Second Embodiment

FIG. 5 shows an image processing apparatus performing image processingbased on the image processing method described in the first embodiment.FIG. 9 shows a specific processing method performed by a defectdetecting apparatus 502.

The image processing apparatus can be used for inspecting an imagecapturing device, such as a digital still camera or digital videocamera, that uses a lens and a CCD (charge-coupled device). Thefollowing is a description of an exemplary application of the imageprocessing apparatus as an inspection apparatus for inspecting lensesand CCDs in a digital still camera to be inspected for defects.

A camera 500 shown in FIG. 5, which is a digital still camera to beinspected, is used to capture an image of an inspection flat panel 501.The flat panel 501 is a solid-pail-color plate, painted gray forexample. An original image P0 is an image of the flat plane 501 capturedby the camera 500, and is digital image data. The original image P0 inthe present embodiment has a brightness of 256 gray levels per pixel ofthe CCD of the camera 500 and a size of 1024×768 pixels. A method fordetecting a defect in a lens or CCD of the camera by using the originalimage P0 on a defect inspection apparatus 502 will be described below.

Reference numeral 503 denotes an image processing unit which implementsan image processing method of the first embodiment described above. Theimage processing unit 503 averages pixels in a 100×100 area. The size ofthis area covered by the image processing unit 503 is large enough forperforming averaging according to the present invention since the numberof pixels in a largest possible defect is on the order of 50×50. Ifaveraging is performed over a 100×100 neighborhood of each of the pixelsin a maximum 50×50 defect portion, the defect portion is equal to ¼ ofthe area at the maximum and at least ¾ of the area has a backgroundbrightness value. Accordingly, the brightness value of the pixels afterthe averaging is much closer to the background brightness than that ofthe defect portion. As a result, the effect of the defect portion ispractically eliminated, and the image processing unit 503 outputs afirst processed image P1 which is very close to the brightnessdistribution of the original image P0.

Reference numeral 504 denotes a subtracter which computes thedifferences between an original image P0 and a first processed image P1outputted from the image processing unit 503. The subtracter 504 outputsa second processed image P2 in which only defect portions, which are thedifferences between the two images, have non-zero values.

The output from the subtracter 504 is then inputted into a binarizingcircuit 505. The binarizing circuit 505 evaluates the output from thesubtracter 504 on a per pixel basis and outputs “1” if the absolutevalue of the brightness of a pixel of interest is greater than apredetermined threshold, or outputs “0” if it is less than or equal tothe threshold. The threshold is set to “1” in this example in order toenable subtle defects to be detected. However, if a defect is clear, agreater threshold may be set. The binarizing circuit 505 outputs “1” fora pixel at which the original image P0 disagrees with the originalbrightness distribution that the original image P0 originally has. As aresult, a resultant image P3 can be obtained in which only defect pixelsare represented as “1”.

The resultant image P3 contains all pixels of defect portions. However,an acceptable-quality standard for some products to be inspected mayaccept products as non-defective, depending on the level of defects.Accordingly, the resultant image P3 should not directly be used to makedetermination as to whether products are defective or not. Therefore, anevaluation value computing circuit 506 and a determining circuit 507evaluate the resultant image P3 and check the evaluation against anallowable-quality standard inputted through a shipping quality rangeinput device 508 to determine whether the quality of the camera 500under inspection is acceptable or not.

For example, if a quality standard is inputted beforehand through theshipping quality range input device 508 that rejects a productsproviding an image containing 50 or more defect pixels in total as adefective one, the determining circuit 507 obtains the total number ofthe pixels having the value “1” in the resultant image P3. If it is lessthan 50, the determining circuit 507 determines that the product isacceptable. If it is greater than or equal to 50, then the determiningcircuit 507 determines that the product is defective. The determiningcircuit 507 outputs the result to a result display device 509. If astandard is inputted beforehand through the shipping quality range inputdevice 508 that rejects a product providing an image in which the numberof the largest set of contiguous defective pixels is 30 or more, thedetermining circuits 507 searches for sets of continuous pixels havingthe value “1” in the resultant image P3 and obtains the number of pixelsin the largest set found. If the number is less than 30, then thedetermining circuit 507 determines that the product is acceptable.Otherwise, it determines that the product is defective. The determiningcircuit 507 outputs the result to the result display device 509. Theresult display device 509 displays the result of determination from thedetermining circuit 507 to indicate to a human operator whether theproduct is acceptable or not.

The output from the subtracter 504 is directly used as the secondprocessed image P2 in the example described above. However, in practice,noise components contained in an original image P0 will have non-zerovalues. In order to remove these, a median filter may be applied to theoutput from the subtracter 504 and the filtered image may be used as thesecond processed image P2. Median filters, which are commonly used inimage processing, sort the brightness values of pixels in a neighborhoodof a pixel of interest in descending order of magnitude and set themiddle value in the series as the brightness value of the pixel ofinterest. The median filter can remove noise portions while preservingonly brightness varying portions having a certain size or larger.

Conventional image processing apparatuses do not guarantee the averagebrightness value in borders. Accordingly, if a conventional imageprocessing method is used in the image processing unit 503, it may failto properly detect defects in a border. In contrast, the present methodcan correctly extract defects because the average value of the bordersof the low-pass filter is close to that in the original image.

As has been described, according to the present embodiment, subtractionis performed between an original image P0 and a first processed image,which is obtained by using the image processing method of the firstembodiment and represents the original brightness distribution of theoriginal image P0, and the differences are binarized. Thus, defects canbe correctly extracted.

An exemplary processing method in which the defect detecting apparatus502 given above is implemented by software will be described below indetail with reference to FIG. 9.

FIG. 9 is a flowchart showing an embodiment of a method for detectingdefects in a lens or CCD during an inspection of a device such as adigital still camera.

At step 901, low-pass filtering described in the first embodiment isapplied to an original image P0 captured by a camera. Here, pixels in a100×100 area will be averaged, which is large enough as compared to alargest possible defect area (50×50). Accordingly, the effects of defectportions are practically removed on the same principles that have beendescribed with respect to the image processing unit 503. The result is afirst processed image P1 which is very close to the brightnessdistribution of the original image P0.

At step 902, the differences between the first processed image P1 andthe original image P0 are obtained. Ideally, only defect portions, whichare the differences between the two images, will have non-zero values asa result of this operation. However, in practice, noise componentscontained in the original image P0 also will have non-zero values.Therefore, at step 903, median filtering is applied to the result of theoperation at step 902.

Median filters, which are commonly used in image processing, sort thebrightness values of pixels in a neighborhood of a pixel of interest indescending order of magnitude and set the middle value in the series asthe brightness value of the pixel of interest. The median filter canremove noise portions while preserving brightness varying portionshaving a certain size or larger. Accordingly, the processing at step 903provides a result P2 in which only defect portions have non-zero values.

At step 904, the result P2 is binarized. In the binarization, the resultP2 is evaluated on a pixel-by-pixel basis and, if the absolute value ofthe brightness of a pixel of interest is greater than a predeterminedthreshold, “1” is outputted for the pixel, or if the absolute value issmaller or equal to the threshold, “0” is outputted for the pixel. Thethreshold is set to “1” in this example in order to enable subtledefects to be detected. It may be set to a greater value if defects areclear. As a result of the binarization, “1” is outputted for a pixel atwhich the brightness of the original image P0 disagrees with theoriginal brightness distribution that the original image P0 originallyhas. Consequently, a resultant image P3 can be obtained in which onlydefect pixels are represented as “1”.

At step 905, the total number of defective pixels contained in theresultant image P3 is counted. This can be accomplished by counting thepixels that have the value “1” in the entire resultant image P3.

At step 906, the count value obtained at step 905 is compared with anacceptable number of defective pixels, for example 50 based on a givenquality standard that “products that provide 50 or more defective pixelsare defective ones”. If the count value obtained at step 905 is greaterthan or equal to 50, information indicating that the item is defectiveis outputted at step 907. On the other hand, if the count value obtainedat step 905 is less than 50, then information indicating that the itemis acceptable is outputted at step 908.

Third Embodiment

FIGS. 6 to 8 show a fluorescent microscope image capturing apparatusperforming image processing according to the image processing method ofthe first embodiment.

In FIG. 6, reference numeral 601 denotes an object to be observed, whichis an image to be corrected and may be fluorescently dyed microbe cells,for example. When a laser beam from a laser light source 602 isreflected by a dichroic mirror and applied to the object to be observed601, the fluorochrome in the object to be observed 601 is energized bythe laser light to emit light with a particular wavelength. The light iscaptured by a CCD camera 604, digitized into image data, and processedin a host computer 605.

It is difficult to uniformly irradiate the observation surface withlaser light emitted to the object to be observed 601. Typically, anillumination distribution is exhibited in which portions closer to thecenter are brighter than portions closer to the edges. If a fluorescentimage energized by the laser light is displayed as is, the actualdistribution of fluorochrome is not accurately displayed. Therefore,calibration of the illumination distribution of the laser light must beperformed.

The calibration is performed by using a calibration image provided inaddition to a captured image of the object to be observed 601, asfollows.

The calibration image is acquired by capturing an image of a standardsample on which a fluorochrome is applied uniformly, instead of anobject to be observed 601 of FIG. 6. Because the acquired calibrationimage represents the illumination distribution of the laser light, eachpixel of the captured image of the object is divided by thecorresponding pixel of the calibration image, thereby to obtain an imageafter calibration in which the effect of the illumination distributionof the laser light is canceled.

However, it is difficult to prepare an ideal standard sample used foracquiring a calibration image. In practice, small contaminations willoften be contained. If a calibration image containing suchcontaminations were directly used for calibration, the contaminationswould appear as shades in a displayed image.

Therefore, in the third embodiment the host computer 605 performs imageprocessing described below.

The host computer 605 includes a calibration image generating routine606 and an observation image correcting routine 607. The calibrationimage generating routine 606, which generate an ideal calibration imagefrom a captured image of a standard sample, is configured as shown inFIG. 7.

At step 701, low-pass filtering is applied only to the pixels in theborders of a captured image of a standard sample.

At step 702, a larger image is generated by extrapolation.

At step 703, low-pass filtering is applied to the region of the originalimage in the generated larger image.

The process so far is the same as in the first embodiment.

Finally, at step 704, the image generated at step 703 is compared withthe original image on a pixel-by-pixel basis and further low-passfiltering is applied to the image generated at step 702 without usingpixels having differences greater than or equal to a predeterminedthreshold.

FIG. 8 details the operation at step 704.

At step 801, the result of step 703 is compared with the brightness ofthe original image on a pixel-by-pixel basis. If the difference inbrightness is greater than or equal to “10”, the pixel in the imageresulting from step 703 is marked with −1 at step 802, indicating thatthe pixel has a significant error. If there is a significant error, thepixel is likely to correspond to a contamination. Therefore, the meaningof the marking is to mark a contamination.

While the threshold is set to “10” or greater in this example, thethreshold may be set to any other value.

The process is performed for every pixel in the original image. If thereremains an unprocessed pixel at step 803, steps 801 to 802 are repeated.Once all pixels having large differences have been marked, the processproceeds to step 804 in order to apply further low-pass filtering.

A pixel is set as the pixel of interest, which is specified in order, atstep 804, and the values of pixels in a neighborhood of the pixel ofinterest which are used in low-pass filtering are checked at step 805.

If the value of a pixel is −1, the pixel is ignored. If not −1, thevalue of the pixel is used in averaging at step 806. For example, ifthere are 25 pixels to be used in a low-pass filter and 2 pixels amongthem have the value −1, the average value of the remaining 23 pixels arecomputed and set as a new brightness value of the pixel of interest. Atstep 807, determination is made as to whether all pixels in theneighborhood of the current pixel of interest that are to be used in thelow-pass filter have been processed. If there remains an unprocessedpixel, steps 805 and 806 are repeated. If it is determined at step 807that all pixels have been processed, the result of low-pass filteringfor the one single pixel is stored at step 808.

At step 809, determination is made as to whether or not the processgiven above has been completed for all pixels. If not, steps 804 to 808are repeated until all pixels are processed, then the process will end.

With this process, an optimum calibration image that has not beenaffected by contaminations can be generated by low-pass filteringwithout using pixels that have values departing from the averagebrightness due to contamination. The generated calibration image isstored in the host computer 605.

After the calibration image has been obtained by executing thecalibration image generating routine 606 on the host computer 605, theoperator replaces the standard sample under observation with an ultimateobject to be observed 601, such as microbe cells dyed with afluorochrome, and instructs the computer 605 to execute the observationimage correcting routine 607. Then, the observation image correctingroutine 607 performs calibration, in which the optimum calibrationimage, not affected by contaminations because pixels having brightnessvalues departing from the average brightness due to contaminations havebeen excluded from the low-pass filtering, is used to perform divisionon a corresponding pixel-by-pixel basis to accurately cancel the effectof illumination distribution of laser light. Thus, the originaldistribution of the fluorochrome can be accurately displayed.

Low-pass filtering has been applied only to the pixels in the borders ofthe original image P0 in the early stage of the process in theembodiments described above. However, the low-pass filtering may beapplied only to pixels near the borders of the original image P0 and thepixels that are nearer to the edges than those pixels may be discarded,and then the subsequent image processing may be performed.

Furthermore, while the low-pass filtering has been applied only to thepixels in the borders or pixels near the borders of the original imageP0 in the early stage of the process in the embodiments, describedabove, the low-pass filtering may also be applied to the remainingportions of the original image P0 at the same time. The intended purposecan be accomplished by applying low-pass filtering to at least pixels inthe borders or pixels near the borders of the original image.

It should be noted that “applying low-pass filtering by using pixelslocated in the direction perpendicular to the direction ofextrapolation” herein has the following meaning: If the pixels C(2, −1)and C(2, −2) are to be vertically extrapolated for the pixels B(2, 0)and A(2, 0), the pixel B(2, 0) is the origin and the pixels B(1, 0) andB(3, 0) located in the horizontal direction are the pixels located inthe direction perpendicular to the vertical extrapolation. If the pixelsC(−1, 2) and C(−2, 2) are to be horizontally extrapolated for pixelsB(0, 2) and A(1, 2), the pixel B(0, 2) is the origin and the pixels B(0,3) and B(0, 1) in the vertical direction are the pixels locatedperpendicular to the horizontal extrapolation.

The image processing method according to the present invention providesan accurate result that guarantees the average brightness in borders,and therefore an accurate low-pass filter can be implemented. Forexample, using an image processing apparatus incorporating this imageprocessing method to inspect a digital camera is useful, because a goodinspection method that is free of erroneous detection can beimplemented.

Furthermore, the image processing method according to the presentinvention is useful because if this method is used for calibration of afluorescent microscope, for example, an accurate calibration can beimplemented and an accurate display can be achieved.

1. An image processing method of treating an image as two-dimensionaldata of pixels having multiple gray levels, the method comprising:applying low-pass filtering to pixels in borders of an original image;extending the region of the original image and obtaining the brightnessof the extended portion by extrapolation using as an origin a pixelincluded in the region to which the low-pass filtering has been applied;and applying low-pass filtering to the region of the original image byusing the brightness of the extended portion.
 2. The image processingmethod according to claim 1, wherein applying low-pass filtering to thepixels in the borders of an original image comprises applyingone-dimensional low-pass filtering to the pixels in the borders of theoriginal image.
 3. The image processing method according to claim 1,wherein applying low-pass filtering to the pixels in the borders of anoriginal image comprises applying low-pass filtering by using pixelslocated in the direction perpendicular to the direction ofextrapolation.
 4. The image processing method according to claim 1,wherein the extent to which the region of the original image is extendedis determined depending on the extent of low-pass filtering to beapplied to the region of the original image by using the brightness ofthe extended portion.
 5. The image processing method according to claim4, wherein if the size of a neighborhood pixel region required forcomputation in low-pass filtering applied to the region of the originalimage including the extended portion is A pixels high x B pixels wide,then the region of the original image is extended by A pixels in thevertical direction and by B pixels in the horizontal direction.
 6. Theimage processing method according to claim 1, further comprisingobtaining differences between a first processed image obtained byperforming the image processing method according to claim 1 and anoriginal image to obtain a second processed image.
 7. The imageprocessing method according to claim 6, wherein the second processedimage is binarized to detect a defective portion contained in theoriginal image.